osi converter: preserve N dimensions over one column across the round-trip (#222)#229
Merged
Conversation
…-trip (#222) OBML allows N dimensions over one column (grain variants, role-playing via `via`); OSI is field-centric (one dimension per field). The converter used to `break` after the first matching dimension on export and build one dimension per field on import, so a second dimension on the same column was dropped silently - both names lost, zero warnings. - Export (obml_to_osi): the first matching dimension stays the primary carried on the field; any extras are serialized into an `obml_extra_dimensions` list extension (name + resultType + timeGrain + format + description + owner + via), and a fidelity warning is emitted (warn, do not raise). - Import (osi_to_obml): rebuild each preserved descriptor as its own dimension. The collision-key logic is factored into `_insert_dimension` and reused for the primary and every extra. Descriptors are opaque foreign-modifiable data, so each is shape-guarded (must be a dict with a non-empty string name; string props only) - a malformed payload is skipped, never crashes on an unhashable key. Tests: TestMultipleDimensionsPerColumn - all dimensions survive with distinct grains, export warns (not silent), and malformed extra-dimension payloads never crash. Closes #222. Refs #201, #202, #220.
The obml_extra_dimensions descriptor only carried the scalar props (resultType, timeGrain, format, description, owner, via), so a non-primary dimension's own synonyms and vendor customExtensions were dropped on the round-trip. Include both in the descriptor on export and restore them on import, with the same shape guards as the rest of the opaque extension data: synonyms kept only as non-empty strings, customExtensions only as dict entries. Malformed values are filtered, never crash. Tests: extra dimension's synonyms + customExtensions survive the round-trip; malformed synonyms/customExtensions payloads are filtered without crashing. Refs #222.
The primary (single) dimension on a column lost its own synonyms entirely and its customExtensions moved to the column (OSI has no dimension entity, so they surfaced as field foreign extensions and re-imported onto the column). Now the extras preserved these but the primary did not - inconsistent. Stash the primary dimension's synonyms and customExtensions in authoritative obml_dimension_synonyms / obml_dimension_custom_extensions extension keys and restore them to the dimension on import, with the same shape guards (string synonyms, dict extensions). The existing foreign-extension emission stays, so the dimension's vendor extensions remain visible to other OSI tools on the field; that column-side surfacing is pre-existing and unchanged. Test: the primary dimension's synonyms + customExtensions survive the round trip. Full package suite green (the field-foreign-extension export test still passes). Refs #222.
ralfbecher
pushed a commit
to ralfbecher/OSI
that referenced
this pull request
Jul 17, 2026
Mirrors two further canonical converter fixes on top of the round-trip robustness work already in this PR: - ralforion/orionbelt-semantic-layer#228: restore the OBML dimension name across the round trip. The OSI field name is the physical column code, so a round trip renamed every dimension to its code and could trip the collision fallback for names unique in the source. The name is now preserved in an obml_dimension_name extension and restored on import, making the collision fallback genuinely foreign-OSI only. Non-string values are ignored so a foreign payload cannot crash the converter on an unhashable key. - ralforion/orionbelt-semantic-layer#229: preserve N dimensions over one column. OBML allows many dimensions on one column (grain variants, role playing); OSI is field-centric. Extras are now preserved in an obml_extra_dimensions list extension with a fidelity warning (warn, not raise) and rebuilt on import. Both primary and extra dimensions carry their own synonyms and vendor extensions, with shape guards on the opaque extension data. 177 converter tests pass; ruff clean.
ralfbecher
added a commit
that referenced
this pull request
Jul 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #222. Builds on #220's extension hook.
Problem
OBML allows N dimensions over one column (grain variants, role-playing via
via); OSI is field-centric (one dimension per field). The converter used tobreakafter the first matching dimension on export and build one dimension per field on import, so a second dimension on the same column was dropped silently - both names lost, zero warnings on either side.Repro (from the issue):
OBML -> OSI -> OBMLused to yield a singleorder_datedimension.Fix
obml_to_osi): the first matching dimension stays the primary carried on the field; any extras are serialized into anobml_extra_dimensionslist extension (name + resultType + timeGrain + format + description + owner + via), and a fidelity warning is emitted (warn, do not raise - conversion still succeeds).osi_to_obml): rebuild each preserved descriptor as its own dimension. The collision-key logic is factored into_insert_dimensionand reused for the primary and every extra.Now the same repro round-trips to all three dimensions with their distinct grains, and the export warns rather than silently dropping.
Robustness
Descriptors are opaque foreign-modifiable data (
validate_osidoes not inspect extension payloads), so each is shape-guarded - must be a dict with a non-empty string name, string props only. A malformed payload (not a list, non-dict items, non-string names) is skipped, never crashes on an unhashable key. This is the same class of guard #228 added forobml_dimension_name, applied here from the start.Tests
TestMultipleDimensionsPerColumn: all dimensions survive with distinct grains and the same physical column; export warns (not silent); parametrized malformed-payload cases never crash.177 package + convert-integration tests pass; ruff clean; no new mypy errors (package's pre-existing typing debt unchanged, outside CI's
mypy src/scope).Upstream note
Whether OSI should natively represent multiple dimensions over one field (cf. LookML
dimension_group, Cubegranularities) remains an Ossie spec question, tracked in the issue. This PR is the converter-side warn + preserve, independent of that outcome.Refs #201, #202, #220.